

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>Manifest &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
        <script src="../../../_static/jquery.js"></script>
        <script src="../../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../../" id="documentation_options" src="../../../_static/documentation_options.js"></script>
        <script src="../../../_static/doctools.js"></script>
        <script src="../../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../../genindex/" />
    <link rel="search" title="Search" href="../../../search/" />
    <link rel="next" title="Map and PG Message handling" href="../map_message_handling/" />
    <link rel="prev" title="Log Based PG" href="../log_based_pg/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../../internals/">Ceph 内幕</a></li>
          <li class="breadcrumb-item"><a href="../">OSD 开发者文档</a></li>
      <li class="breadcrumb-item active">Manifest</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../../_sources/dev/osd_internals/manifest.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../developer_guide/">开发者指南</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../../internals/">Ceph 内幕</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../../balancer-design/">Ceph 如何均衡（读写、容量）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../blkin/">Tracing Ceph With LTTng</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../blkin/#tracing-ceph-with-blkin">Tracing Ceph With Blkin</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../bluestore/">BlueStore Internals</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../ceph_krb_auth/">如何配置好 Ceph Kerberos 认证的详细文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../cephfs-mirroring/">CephFS Mirroring</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../cephfs-reclaim/">CephFS Reclaim Interface</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../cephfs-snapshots/">CephFS 快照</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../cephx/">Cephx</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../cephx_protocol/">Cephx 认证协议详细阐述</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../config/">配置管理系统</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../config-key/">config-key layout</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../context/">CephContext</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../continuous-integration/">Continuous Integration Architecture</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../corpus/">资料库结构</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../cpu-profiler/">Oprofile 的安装</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../crush-msr/">CRUSH MSR (Multi-step Retry)</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../cxx/">C++17 and libstdc++ ABI</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../deduplication/">去重</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../delayed-delete/">CephFS delayed deletion</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../dev_cluster_deployment/">开发集群的部署</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../dev_cluster_deployment/#id5">在同一机器上部署多套开发集群</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../development-workflow/">开发流程</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../documenting/">为 Ceph 写作文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../dpdk/">Ceph messenger DPDKStack</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../encoding/">序列化（编码、解码）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../erasure-coded-pool/">纠删码存储池</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../file-striping/">File striping</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../freebsd/">FreeBSD Implementation details</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../generatedocs/">Ceph 文档的构建</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../health-reports/">Health Reports</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../iana/">IANA 号</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../kclient/">Testing changes to the Linux Kernel CephFS driver</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../kclient/#step-one-build-the-kernel">Step One: build the kernel</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../kclient/#step-two-create-a-vm">Step Two: create a VM</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../kclient/#step-three-networking-the-vm">Step Three: Networking the VM</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../kubernetes/">Hacking on Ceph in Kubernetes with Rook</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../libcephfs_proxy/">Design of the libcephfs proxy</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../libs/">库体系结构</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../logging/">集群日志的用法</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../logs/">调试日志</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../macos/">在 MacOS 上构建</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mempool_accounting/">What is a mempool?</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mempool_accounting/#some-common-mempools-that-we-can-track">Some common mempools that we can track</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../messenger/">Messenger notes</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mon-bootstrap/">Monitor bootstrap</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mon-elections/">Monitor Elections</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mon-on-disk-formats/">ON-DISK FORMAT</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../mon-osdmap-prune/">FULL OSDMAP VERSION PRUNING</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../msgr2/">msgr2 协议（ msgr2.0 和 msgr2.1 ）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../network-encoding/">Network Encoding</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../network-protocol/">网络协议</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../object-store/">对象存储架构概述</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../osd-class-path/">OSD class path issues</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../peering/">互联</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../perf/">Using perf</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../perf_counters/">性能计数器</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../perf_histograms/">Perf histograms</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../placement-group/">PG （归置组）说明</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../quick_guide/">开发者指南（快速）</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../rados-client-protocol/">RADOS 客户端协议</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../rbd-diff/">RBD 增量备份</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../rbd-export/">RBD Export &amp; Import</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../rbd-layering/">RBD Layering</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../release-checklists/">Release checklists</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../release-process/">Ceph Release Process</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../seastore/">SeaStore</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../sepia/">Sepia 社区测试实验室</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../session_authentication/">Session Authentication for the Cephx Protocol</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../testing/">测试笔记</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../versions/">Public OSD Version</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../vstart-ganesha/">NFS CephFS-RGW Developer Guide</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../wireshark/">Wireshark Dissector</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../zoned-storage/">Zoned Storage Support</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../">OSD 开发者文档</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../async_recovery/">异步恢复</a></li>
<li class="toctree-l3"><a class="reference internal" href="../backfill_reservation/">Backfill Reservation</a></li>
<li class="toctree-l3"><a class="reference internal" href="../erasure_coding/">纠删码编码的归置组</a></li>
<li class="toctree-l3"><a class="reference internal" href="../last_epoch_started/">last_epoch_started</a></li>
<li class="toctree-l3"><a class="reference internal" href="../log_based_pg/">Log Based PG</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">Manifest</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#introduction">Introduction</a></li>
<li class="toctree-l4"><a class="reference internal" href="#intended-usage-model">Intended Usage Model</a></li>
<li class="toctree-l4"><a class="reference internal" href="#status-and-future-work">Status and Future Work</a></li>
<li class="toctree-l4"><a class="reference internal" href="#data-structures">Data Structures</a></li>
<li class="toctree-l4"><a class="reference internal" href="#request-handling">Request Handling</a></li>
<li class="toctree-l4"><a class="reference internal" href="#rados-interface">RADOS Interface</a></li>
<li class="toctree-l4"><a class="reference internal" href="#ceph-dedup-tool">ceph-dedup-tool</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="../map_message_handling/">Map and PG Message handling</a></li>
<li class="toctree-l3"><a class="reference internal" href="../mclock_wpq_cmp_study/">QoS Study with mClock and WPQ Schedulers</a></li>
<li class="toctree-l3"><a class="reference internal" href="../osd_overview/">OSD</a></li>
<li class="toctree-l3"><a class="reference internal" href="../partial_object_recovery/">Partial Object Recovery</a></li>
<li class="toctree-l3"><a class="reference internal" href="../past_intervals/">OSDMap Trimming and PastIntervals</a></li>
<li class="toctree-l3"><a class="reference internal" href="../pg/">PG</a></li>
<li class="toctree-l3"><a class="reference internal" href="../pg_removal/">PG Removal</a></li>
<li class="toctree-l3"><a class="reference internal" href="../pgpool/">PGPool</a></li>
<li class="toctree-l3"><a class="reference internal" href="../recovery_reservation/">Recovery Reservation</a></li>
<li class="toctree-l3"><a class="reference internal" href="../refcount/">Refcount</a></li>
<li class="toctree-l3"><a class="reference internal" href="../scrub/">Scrub internals and diagnostics</a></li>
<li class="toctree-l3"><a class="reference internal" href="../snaps/">快照</a></li>
<li class="toctree-l3"><a class="reference internal" href="../stale_read/">Preventing Stale Reads</a></li>
<li class="toctree-l3"><a class="reference internal" href="../watch_notify/">关注通知</a></li>
<li class="toctree-l3"><a class="reference internal" href="../wbthrottle/">回写抑制</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../../mds_internals/">MDS 开发者文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../radosgw/">RADOS 网关开发者文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../ceph-volume/">ceph-volume 开发者文档</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../crimson/">Crimson developer documentation</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="manifest">
<h1>Manifest<a class="headerlink" href="#manifest" title="Permalink to this heading"></a></h1>
<section id="introduction">
<h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this heading"></a></h2>
<p>As described in <code class="docutils literal notranslate"><span class="pre">../deduplication.rst</span></code>, adding transparent redirect
machinery to RADOS would enable a more capable tiering solution
than RADOS currently has with “cache/tiering”.</p>
<p>See <code class="docutils literal notranslate"><span class="pre">../deduplication.rst</span></code></p>
<p>At a high level, each object has a piece of metadata embedded in
the <code class="docutils literal notranslate"><span class="pre">object_info_t</span></code> which can map subsets of the object data payload
to (refcounted) objects in other pools.</p>
<p>This document exists to detail:</p>
<ol class="arabic simple">
<li><p>Manifest data structures</p></li>
<li><p>Rados operations for manipulating manifests.</p></li>
<li><p>Status and Plans</p></li>
</ol>
</section>
<section id="intended-usage-model">
<h2>Intended Usage Model<a class="headerlink" href="#intended-usage-model" title="Permalink to this heading"></a></h2>
<section id="rbd">
<h3>RBD<a class="headerlink" href="#rbd" title="Permalink to this heading"></a></h3>
<p>For RBD, the primary goal is for either an OSD-internal agent or a
cluster-external agent to be able to transparently shift portions
of the consituent 4MB extents between a dedup pool and a hot base
pool.</p>
<p>As such, RBD operations (including class operations and snapshots)
must have the same observable results regardless of the current
status of the object.</p>
<p>Moreover, tiering/dedup operations must interleave with RBD operations
without changing the result.</p>
<p>Thus, here is a sketch of how I’d expect a tiering agent to perform
basic operations:</p>
<ul>
<li><p>Demote cold RBD chunk to slow pool:</p>
<ol class="arabic simple">
<li><p>Read object, noting current user_version.</p></li>
<li><p>In memory, run CDC implementation to fingerprint object.</p></li>
<li><p>Write out each resulting extent to an object in the cold pool
using the CAS class.</p></li>
<li><p>Submit operation to base pool:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">ASSERT_VER</span></code> with the user version from the read to fail if the
object has been mutated since the read.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code> for each of the extents to the corresponding object
in the base pool.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">EVICT_CHUNK</span></code> for each extent to free up space in the base pool.
Results in each chunk being marked <code class="docutils literal notranslate"><span class="pre">MISSING</span></code>.</p></li>
</ul>
</li>
</ol>
<p>RBD users should then either see the state prior to the demotion or
subsequent to it.</p>
<p>Note that between 3 and 4, we potentially leak references, so a
periodic scrub would be needed to validate refcounts.</p>
</li>
<li><p>Promote cold RBD chunk to fast pool.</p>
<ol class="arabic simple">
<li><p>Submit <code class="docutils literal notranslate"><span class="pre">TIER_PROMOTE</span></code></p></li>
</ol>
</li>
</ul>
<p>For clones, all of the above would be identical except that the
initial read would need a <code class="docutils literal notranslate"><span class="pre">LIST_SNAPS</span></code> to determine which clones exist
and the <code class="docutils literal notranslate"><span class="pre">PROMOTE</span></code> or <code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code>/<code class="docutils literal notranslate"><span class="pre">EVICT</span></code> operations would need to include
the <code class="docutils literal notranslate"><span class="pre">cloneid</span></code>.</p>
</section>
<section id="radosgw">
<h3>RadosGW<a class="headerlink" href="#radosgw" title="Permalink to this heading"></a></h3>
<p>For reads, RADOS Gateway (RGW) could operate as RBD does above relying on the
manifest machinery in the OSD to hide the distinction between the object
being dedup’d or present in the base pool</p>
<p>For writes, RGW could operate as RBD does above, but could
optionally have the freedom to fingerprint prior to doing the write.
In that case, it could immediately write out the target objects to the
CAS pool and then atomically write an object with the corresponding
chunks set.</p>
</section>
</section>
<section id="status-and-future-work">
<h2>Status and Future Work<a class="headerlink" href="#status-and-future-work" title="Permalink to this heading"></a></h2>
<p>At the moment, initial versions of a manifest data structure along
with IO path support and rados control operations exist.  This section
is meant to outline next steps.</p>
<p>At a high level, our future work plan is:</p>
<ul class="simple">
<li><p>Cleanups: Address immediate inconsistencies and shortcomings outlined
in the next section.</p></li>
<li><p>Testing: Rados relies heavily on teuthology failure testing to validate
features like cache/tiering.  We’ll need corresponding tests for
manifest operations.</p></li>
<li><p>Snapshots: We want to be able to deduplicate portions of clones
below the level of the rados snapshot system.  As such, the
rados operations below need to be extended to work correctly on
clones (e.g.: we should be able to call <code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code> on a clone, clear the
corresponding extent in the base pool, and correctly maintain OSD metadata).</p></li>
<li><p>Cache/tiering: Ultimately, we’d like to be able to deprecate the existing
cache/tiering implementation, but to do that we need to ensure that we
can address the same use cases.</p></li>
</ul>
<section id="cleanups">
<h3>Cleanups<a class="headerlink" href="#cleanups" title="Permalink to this heading"></a></h3>
<p>The existing implementation has some things that need to be cleaned up:</p>
<ul>
<li><p><code class="docutils literal notranslate"><span class="pre">SET_REDIRECT</span></code>: Should create the object if it doesn’t exist, otherwise
one couldn’t create an object atomically as a redirect.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code>:</p>
<ul>
<li><p>Appears to trigger a new clone as user_modify gets set in
<code class="docutils literal notranslate"><span class="pre">do_osd_ops</span></code>.  This probably isn’t desirable, see Snapshots section
below for some options on how generally to mix these operations
with snapshots.  At a minimum, <code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code> probably shouldn’t set
user_modify.</p></li>
<li><p>Appears to assume that the corresponding section of the object
does not exist (sets <code class="docutils literal notranslate"><span class="pre">FLAG_MISSING</span></code>) but does not check whether the
corresponding extent exists already in the object.  Should always
leave the extent clean.</p></li>
<li><p>Appears to clear the manifest unconditionally if not chunked,
that’s probably wrong.  We should return an error if it’s a
<code class="docutils literal notranslate"><span class="pre">REDIRECT</span></code></p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">case</span> <span class="n">CEPH_OSD_OP_SET_CHUNK</span><span class="p">:</span>
  <span class="k">if</span> <span class="p">(</span><span class="n">oi</span><span class="o">.</span><span class="n">manifest</span><span class="o">.</span><span class="n">is_redirect</span><span class="p">())</span> <span class="p">{</span>
    <span class="n">result</span> <span class="o">=</span> <span class="o">-</span><span class="n">EINVAL</span><span class="p">;</span>
    <span class="n">goto</span> <span class="n">fail</span><span class="p">;</span>
  <span class="p">}</span>
</pre></div>
</div>
</li>
</ul>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">TIER_PROMOTE</span></code>:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">SET_REDIRECT</span></code> clears the contents of the object.  <code class="docutils literal notranslate"><span class="pre">PROMOTE</span></code> appears
to copy them back in, but does not unset the redirect or clear the
reference. This violates the invariant that a redirect object
should be empty in the base pool.  In particular, as long as the
redirect is set, it appears that all operations will be proxied
even after the promote defeating the purpose.  We do want <code class="docutils literal notranslate"><span class="pre">PROMOTE</span></code>
to be able to atomically replace a redirect with the actual
object, so the solution is to clear the redirect at the end of the
promote.</p></li>
<li><p>For a chunked manifest, we appear to flush prior to promoting.
Promotion will often be used to prepare an object for low latency
reads and writes, accordingly, the only effect should be to read
any <code class="docutils literal notranslate"><span class="pre">MISSING</span></code> extents into the base pool.  No flushing should be done.</p></li>
</ul>
</li>
<li><p>High Level:</p>
<ul class="simple">
<li><p>It appears that <code class="docutils literal notranslate"><span class="pre">FLAG_DIRTY</span></code> should never be used for an extent pointing
at a dedup extent.  Writing the mutated extent back to the dedup pool
requires writing a new object since the previous one cannot be mutated,
just as it would if it hadn’t been dedup’d yet.  Thus, we should always
drop the reference and remove the manifest pointer.</p></li>
<li><p>There isn’t currently a way to “evict” an object region.  With the above
change to <code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code> to always retain the existing object region, we
need an <code class="docutils literal notranslate"><span class="pre">EVICT_CHUNK</span></code> operation to then remove the extent.</p></li>
</ul>
</li>
</ul>
</section>
<section id="testing">
<h3>Testing<a class="headerlink" href="#testing" title="Permalink to this heading"></a></h3>
<p>We rely really heavily on randomized failure testing.  As such, we need
to extend that testing to include dedup/manifest support as well.  Here’s
a short list of the touchpoints:</p>
<ul>
<li><p>Thrasher tests like <code class="docutils literal notranslate"><span class="pre">qa/suites/rados/thrash/workloads/cache-snaps.yaml</span></code></p>
<p>That test, of course, tests the existing cache/tiering machinery.  Add
additional files to that directory that instead setup a dedup pool.  Add
support to <code class="docutils literal notranslate"><span class="pre">ceph_test_rados</span></code> (<code class="docutils literal notranslate"><span class="pre">src/test/osd/TestRados*</span></code>).</p>
</li>
<li><p>RBD tests</p>
<p>Add a test that runs an RBD workload concurrently with blind
promote/evict operations.</p>
</li>
<li><p>RGW</p>
<p>Add a test that runs a rgw workload concurrently with blind
promote/evict operations.</p>
</li>
</ul>
</section>
<section id="snapshots">
<h3>Snapshots<a class="headerlink" href="#snapshots" title="Permalink to this heading"></a></h3>
<p>Fundamentally we need to be able to manipulate the manifest
status of clones because we want to be able to dynamically promote,
flush (if the state was dirty when the clone was created), and evict
extents from clones.</p>
<p>As such, the plan is to allow the <code class="docutils literal notranslate"><span class="pre">object_manifest_t</span></code> for each clone
to be independent.  Here’s an incomplete list of the high level
tasks:</p>
<ul class="simple">
<li><p>Modify the op processing pipeline to permit <code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code>, <code class="docutils literal notranslate"><span class="pre">EVICT_CHUNK</span></code>
to operation directly on clones.</p></li>
<li><p>Ensure that recovery checks the object_manifest prior to trying to
use the overlaps in clone_range.  <code class="docutils literal notranslate"><span class="pre">ReplicatedBackend::calc_*_subsets</span></code>
are the two methods that would likely need to be modified.</p></li>
</ul>
<p>See <code class="docutils literal notranslate"><span class="pre">snaps.rst</span></code> for a rundown of the <code class="docutils literal notranslate"><span class="pre">librados</span></code> snapshot system and OSD
support details.  I’d like to call out one particular data structure
we may want to exploit.</p>
<p>The dedup-tool needs to be updated to use <code class="docutils literal notranslate"><span class="pre">LIST_SNAPS</span></code> to discover
clones as part of leak detection.</p>
<p>An important question is how we deal with the fact that many clones
will frequently have references to the same backing chunks at the same
offset.  In particular, <code class="docutils literal notranslate"><span class="pre">make_writeable</span></code> will generally create a clone
that shares the same <code class="docutils literal notranslate"><span class="pre">object_manifest_t</span></code> references with the exception
of any extents modified in that transaction.  The metadata that
commits as part of that transaction must therefore map onto the same
refcount as before because otherwise we’d have to first increment
refcounts on backing objects (or risk a reference to a dead object)
Thus, we introduce a simple convention: consecutive clones which
share a reference at the same offset share the same refcount.  This
means that a write that invokes <code class="docutils literal notranslate"><span class="pre">make_writeable</span></code> may decrease refcounts,
but not increase them.  This has some conquences for removing clones.
Consider the following sequence</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>write foo [0, 1024)
flush foo -&gt;
  head: [0, 512) aaa, [512, 1024) bbb
  refcount(aaa)=1, refcount(bbb)=1
snapshot 10
write foo [0, 512) -&gt;
  head:               [512, 1024) bbb
  10  : [0, 512) aaa, [512, 1024) bbb
  refcount(aaa)=1, refcount(bbb)=1
flush foo -&gt;
  head: [0, 512) ccc, [512, 1024) bbb
  10  : [0, 512) aaa, [512, 1024) bbb
  refcount(aaa)=1, refcount(bbb)=1, refcount(ccc)=1
snapshot 20
write foo [0, 512) (same contents as the original write)
  head:               [512, 1024) bbb
  20  : [0, 512) ccc, [512, 1024) bbb
  10  : [0, 512) aaa, [512, 1024) bbb
  refcount(aaa)=?, refcount(bbb)=1
flush foo
  head: [0, 512) aaa, [512, 1024) bbb
  20  : [0, 512) ccc, [512, 1024) bbb
  10  : [0, 512) aaa, [512, 1024) bbb
  refcount(aaa)=?, refcount(bbb)=1, refcount(ccc)=1
</pre></div>
</div>
<p>What should be the refcount for <code class="docutils literal notranslate"><span class="pre">aaa</span></code> be at the end?  By our
above rule, it should be <code class="docutils literal notranslate"><span class="pre">2</span></code> since the two <code class="docutils literal notranslate"><span class="pre">`aaa`</span></code> refs are not
contiguous.  However, consider removing clone <code class="docutils literal notranslate"><span class="pre">20</span></code></p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>initial:
  head: [0, 512) aaa, [512, 1024) bbb
  20  : [0, 512) ccc, [512, 1024) bbb
  10  : [0, 512) aaa, [512, 1024) bbb
  refcount(aaa)=2, refcount(bbb)=1, refcount(ccc)=1
trim 20
  head: [0, 512) aaa, [512, 1024) bbb
  10  : [0, 512) aaa, [512, 1024) bbb
  refcount(aaa)=?, refcount(bbb)=1, refcount(ccc)=0
</pre></div>
</div>
<p>At this point, our rule dictates that <code class="docutils literal notranslate"><span class="pre">refcount(aaa)</span></code> is <cite>1</cite>.
This means that removing <code class="docutils literal notranslate"><span class="pre">20</span></code> needs to check for refs held by
the clones on either side which will then match.</p>
<p>See <code class="docutils literal notranslate"><span class="pre">osd_types.h:object_manifest_t::calc_refs_to_drop_on_removal</span></code>
for the logic implementing this rule.</p>
<p>This seems complicated, but it gets us two valuable properties:</p>
<ol class="arabic simple">
<li><p>The refcount change from make_writeable will not block on
incrementing a ref</p></li>
<li><p>We don’t need to load the <code class="docutils literal notranslate"><span class="pre">object_manifest_t</span></code> for every clone
to determine how to handle removing one -- just the ones
immediately preceding and succeeding it.</p></li>
</ol>
<p>All clone operations will need to consider adjacent <code class="docutils literal notranslate"><span class="pre">chunk_maps</span></code>
when adding or removing references.</p>
</section>
<section id="cache-tiering">
<h3>Cache/Tiering<a class="headerlink" href="#cache-tiering" title="Permalink to this heading"></a></h3>
<p>There already exists a cache/tiering mechanism based on whiteouts.
One goal here should ultimately be for this manifest machinery to
provide a complete replacement.</p>
<p>See <code class="docutils literal notranslate"><span class="pre">cache-pool.rst</span></code></p>
<p>The manifest machinery already shares some code paths with the
existing cache/tiering code, mainly <code class="docutils literal notranslate"><span class="pre">stat_flush</span></code>.</p>
<p>In no particular order, here’s in incomplete list of things that need
to be wired up to provide feature parity:</p>
<ul class="simple">
<li><p>Online object access information: The osd already has pool configs
for maintaining bloom filters which provide estimates of access
recency for objects.  We probably need to modify this to permit
hitset maintenance for a normal pool -- there are already
<code class="docutils literal notranslate"><span class="pre">CEPH_OSD_OP_PG_HITSET*</span></code> interfaces for querying them.</p></li>
<li><p>Tiering agent: The osd already has a background tiering agent which
would need to be modified to instead flush and evict using
manifests.</p></li>
<li><p>Use exiting existing features regarding the cache flush policy such as
histset, age, ratio.
- hitset
- age, ratio, bytes</p></li>
<li><p>Add tiering-mode to <code class="docutils literal notranslate"><span class="pre">manifest-tiering</span></code>
- Writeback
- Read-only</p></li>
</ul>
</section>
</section>
<section id="data-structures">
<h2>Data Structures<a class="headerlink" href="#data-structures" title="Permalink to this heading"></a></h2>
<p>Each RADOS object contains an <code class="docutils literal notranslate"><span class="pre">object_manifest_t</span></code> embedded within the
<code class="docutils literal notranslate"><span class="pre">object_info_t</span></code> (see <code class="docutils literal notranslate"><span class="pre">osd_types.h</span></code>):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">struct</span> <span class="n">object_manifest_t</span> <span class="p">{</span>
        <span class="n">enum</span> <span class="p">{</span>
                <span class="n">TYPE_NONE</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span>
                <span class="n">TYPE_REDIRECT</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span>
                <span class="n">TYPE_CHUNKED</span> <span class="o">=</span> <span class="mi">2</span><span class="p">,</span>
        <span class="p">};</span>
        <span class="n">uint8_t</span> <span class="nb">type</span><span class="p">;</span>  <span class="o">//</span> <span class="n">redirect</span><span class="p">,</span> <span class="n">chunked</span><span class="p">,</span> <span class="o">...</span>
        <span class="n">hobject_t</span> <span class="n">redirect_target</span><span class="p">;</span>
        <span class="n">std</span><span class="p">::</span><span class="nb">map</span><span class="o">&lt;</span><span class="n">uint64_t</span><span class="p">,</span> <span class="n">chunk_info_t</span><span class="o">&gt;</span> <span class="n">chunk_map</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">type</span></code> enum reflects three possible states an object can be in:</p>
<ol class="arabic simple">
<li><p><code class="docutils literal notranslate"><span class="pre">TYPE_NONE</span></code>: normal RADOS object</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">TYPE_REDIRECT</span></code>: object payload is backed by a single object
specified by <code class="docutils literal notranslate"><span class="pre">redirect_target</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">TYPE_CHUNKED:</span> <span class="pre">object</span> <span class="pre">payload</span> <span class="pre">is</span> <span class="pre">distributed</span> <span class="pre">among</span> <span class="pre">objects</span> <span class="pre">with</span>
<span class="pre">size</span> <span class="pre">and</span> <span class="pre">offset</span> <span class="pre">specified</span> <span class="pre">by</span> <span class="pre">the</span> <span class="pre">``chunk_map</span></code>. <code class="docutils literal notranslate"><span class="pre">chunk_map</span></code> maps
the offset of the chunk to a <code class="docutils literal notranslate"><span class="pre">chunk_info_t</span></code> as shown below, also
specifying the <code class="docutils literal notranslate"><span class="pre">length</span></code>, target <cite>OID</cite>, and <code class="docutils literal notranslate"><span class="pre">flags</span></code>.</p></li>
</ol>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">struct</span> <span class="n">chunk_info_t</span> <span class="p">{</span>
  <span class="n">typedef</span> <span class="n">enum</span> <span class="p">{</span>
    <span class="n">FLAG_DIRTY</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span>
    <span class="n">FLAG_MISSING</span> <span class="o">=</span> <span class="mi">2</span><span class="p">,</span>
    <span class="n">FLAG_HAS_REFERENCE</span> <span class="o">=</span> <span class="mi">4</span><span class="p">,</span>
    <span class="n">FLAG_HAS_FINGERPRINT</span> <span class="o">=</span> <span class="mi">8</span><span class="p">,</span>
  <span class="p">}</span> <span class="n">cflag_t</span><span class="p">;</span>
  <span class="n">uint32_t</span> <span class="n">offset</span><span class="p">;</span>
  <span class="n">uint32_t</span> <span class="n">length</span><span class="p">;</span>
  <span class="n">hobject_t</span> <span class="n">oid</span><span class="p">;</span>
  <span class="n">cflag_t</span> <span class="n">flags</span><span class="p">;</span>   <span class="o">//</span> <span class="n">FLAG_</span><span class="o">*</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">FLAG_DIRTY</span></code> at this time can happen if an extent with a fingerprint
is written.  This should be changed to drop the fingerprint instead.</p>
</section>
<section id="request-handling">
<h2>Request Handling<a class="headerlink" href="#request-handling" title="Permalink to this heading"></a></h2>
<p>Similarly to cache/tiering, the initial touchpoint is
<code class="docutils literal notranslate"><span class="pre">maybe_handle_manifest_detail</span></code>.</p>
<p>For manifest operations listed below, we return <code class="docutils literal notranslate"><span class="pre">NOOP</span></code> and continue onto
dedicated handling within <code class="docutils literal notranslate"><span class="pre">do_osd_ops</span></code>.</p>
<p>For redirect objects which haven’t been promoted (apparently <code class="docutils literal notranslate"><span class="pre">oi.size</span> <span class="pre">&gt;</span>
<span class="pre">0</span></code> indicates that it’s present?) we proxy reads and writes.</p>
<p>For reads on <code class="docutils literal notranslate"><span class="pre">TYPE_CHUNKED</span></code>, if <code class="docutils literal notranslate"><span class="pre">can_proxy_chunked_read</span></code> (basically, all
of the ops are reads of extents in the <code class="docutils literal notranslate"><span class="pre">object_manifest_t</span> <span class="pre">chunk_map</span></code>),
we proxy requests to those objects.</p>
</section>
<section id="rados-interface">
<h2>RADOS Interface<a class="headerlink" href="#rados-interface" title="Permalink to this heading"></a></h2>
<p>To set up deduplication one must provision two pools. One will act as the
base pool and the other will act as the chunk pool. The base pool need to be
configured with the <code class="docutils literal notranslate"><span class="pre">fingerprint_algorithm</span></code> option as follows.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>ceph osd pool set $BASE_POOL fingerprint_algorithm sha1|sha256|sha512
--yes-i-really-mean-it
</pre></div>
</div>
<p>Create objects</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">base_pool</span> <span class="n">put</span> <span class="n">foo</span> <span class="o">./</span><span class="n">foo</span>
<span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">chunk_pool</span> <span class="n">put</span> <span class="n">foo</span><span class="o">-</span><span class="n">chunk</span> <span class="o">./</span><span class="n">foo</span><span class="o">-</span><span class="n">chunk</span>
</pre></div>
</div>
<p>Make a manifest object</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>rados -p base_pool set-chunk foo $START_OFFSET $END_OFFSET --target-pool chunk_pool foo-chunk $START_OFFSET --with-reference
</pre></div>
</div>
<p>Operations:</p>
<ul>
<li><p><code class="docutils literal notranslate"><span class="pre">set-redirect</span></code></p>
<p>Set a redirection between a <code class="docutils literal notranslate"><span class="pre">base_object</span></code> in the <code class="docutils literal notranslate"><span class="pre">base_pool</span></code> and a <code class="docutils literal notranslate"><span class="pre">target_object</span></code>
in the <code class="docutils literal notranslate"><span class="pre">target_pool</span></code>.
A redirected object will forward all operations from the client to the
<code class="docutils literal notranslate"><span class="pre">target_object</span></code>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">void</span> <span class="n">set_redirect</span><span class="p">(</span><span class="n">const</span> <span class="n">std</span><span class="p">::</span><span class="n">string</span><span class="o">&amp;</span> <span class="n">tgt_obj</span><span class="p">,</span> <span class="n">const</span> <span class="n">IoCtx</span><span class="o">&amp;</span> <span class="n">tgt_ioctx</span><span class="p">,</span>
              <span class="n">uint64_t</span> <span class="n">tgt_version</span><span class="p">,</span> <span class="nb">int</span> <span class="n">flag</span> <span class="o">=</span> <span class="mi">0</span><span class="p">);</span>

<span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">base_pool</span> <span class="nb">set</span><span class="o">-</span><span class="n">redirect</span> <span class="o">&lt;</span><span class="n">base_object</span><span class="o">&gt;</span> <span class="o">--</span><span class="n">target</span><span class="o">-</span><span class="n">pool</span> <span class="o">&lt;</span><span class="n">target_pool</span><span class="o">&gt;</span>
 <span class="o">&lt;</span><span class="n">target_object</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>Returns <code class="docutils literal notranslate"><span class="pre">ENOENT</span></code> if the object does not exist (TODO: why?)
Returns <code class="docutils literal notranslate"><span class="pre">EINVAL</span></code> if the object already is a redirect.</p>
<p>Takes a reference to target as part of operation, can possibly leak a ref
if the acting set resets and the client dies between taking the ref and
recording the redirect.</p>
<p>Truncates object, clears omap, and clears xattrs as a side effect.</p>
<p>At the top of <code class="docutils literal notranslate"><span class="pre">do_osd_ops</span></code>, does not set user_modify.</p>
<p>This operation is not a user mutation and does not trigger a clone to be created.</p>
<p>There are two purposes of <code class="docutils literal notranslate"><span class="pre">set_redirect</span></code>:</p>
<ol class="arabic simple">
<li><p>Redirect all operation to the target object (like proxy)</p></li>
<li><p>Cache when <code class="docutils literal notranslate"><span class="pre">tier_promote</span></code> is called (redirect will be cleared at this time).</p></li>
</ol>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">set-chunk</span></code></p>
<p>Set the <code class="docutils literal notranslate"><span class="pre">chunk-offset</span></code> in a <code class="docutils literal notranslate"><span class="pre">source_object</span></code> to make a link between it and a
<code class="docutils literal notranslate"><span class="pre">target_object</span></code>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">void</span> <span class="n">set_chunk</span><span class="p">(</span><span class="n">uint64_t</span> <span class="n">src_offset</span><span class="p">,</span> <span class="n">uint64_t</span> <span class="n">src_length</span><span class="p">,</span> <span class="n">const</span> <span class="n">IoCtx</span><span class="o">&amp;</span> <span class="n">tgt_ioctx</span><span class="p">,</span>
           <span class="n">std</span><span class="p">::</span><span class="n">string</span> <span class="n">tgt_oid</span><span class="p">,</span> <span class="n">uint64_t</span> <span class="n">tgt_offset</span><span class="p">,</span> <span class="nb">int</span> <span class="n">flag</span> <span class="o">=</span> <span class="mi">0</span><span class="p">);</span>

<span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">base_pool</span> <span class="nb">set</span><span class="o">-</span><span class="n">chunk</span> <span class="o">&lt;</span><span class="n">source_object</span><span class="o">&gt;</span> <span class="o">&lt;</span><span class="n">offset</span><span class="o">&gt;</span> <span class="o">&lt;</span><span class="n">length</span><span class="o">&gt;</span> <span class="o">--</span><span class="n">target</span><span class="o">-</span><span class="n">pool</span>
 <span class="o">&lt;</span><span class="n">caspool</span><span class="o">&gt;</span> <span class="o">&lt;</span><span class="n">target_object</span><span class="o">&gt;</span> <span class="o">&lt;</span><span class="n">target</span><span class="o">-</span><span class="n">offset</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>Returns <code class="docutils literal notranslate"><span class="pre">ENOENT</span></code> if the object does not exist (TODO: why?)
Returns <code class="docutils literal notranslate"><span class="pre">EINVAL</span></code> if the object already is a redirect.
Returns <code class="docutils literal notranslate"><span class="pre">EINVAL</span></code> if on ill-formed parameter buffer.
Returns <code class="docutils literal notranslate"><span class="pre">ENOTSUPP</span></code> if existing mapped chunks overlap with new chunk mapping.</p>
<p>Takes references to targets as part of operation, can possibly leak refs
if the acting set resets and the client dies between taking the ref and
recording the redirect.</p>
<p>Truncates object, clears omap, and clears xattrs as a side effect.</p>
<p>This operation is not a user mutation and does not trigger a clone to be created.</p>
<p>TODO: <code class="docutils literal notranslate"><span class="pre">SET_CHUNK</span></code> appears to clear the manifest unconditionally if it’s not chunked.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>if (!oi.manifest.is_chunked()) {
  oi.manifest.clear();
}
</pre></div>
</div>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">evict-chunk</span></code></p>
<p>Clears an extent from an object leaving only the manifest link between
it and the <code class="docutils literal notranslate"><span class="pre">target_object</span></code>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">void</span> <span class="n">evict_chunk</span><span class="p">(</span>
  <span class="n">uint64_t</span> <span class="n">offset</span><span class="p">,</span> <span class="n">uint64_t</span> <span class="n">length</span><span class="p">,</span> <span class="nb">int</span> <span class="n">flag</span> <span class="o">=</span> <span class="mi">0</span><span class="p">);</span>

<span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">base_pool</span> <span class="n">evict</span><span class="o">-</span><span class="n">chunk</span> <span class="o">&lt;</span><span class="n">offset</span><span class="o">&gt;</span> <span class="o">&lt;</span><span class="n">length</span><span class="o">&gt;</span> <span class="o">&lt;</span><span class="nb">object</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>Returns <code class="docutils literal notranslate"><span class="pre">EINVAL</span></code> if the extent is not present in the manifest.</p>
<p>Note: this does not exist yet.</p>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">tier-promote</span></code></p>
<p>Promotes the object ensuring that subsequent reads and writes will be local</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">void</span> <span class="n">tier_promote</span><span class="p">();</span>

<span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">base_pool</span> <span class="n">tier</span><span class="o">-</span><span class="n">promote</span> <span class="o">&lt;</span><span class="n">obj</span><span class="o">-</span><span class="n">name</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>Returns <code class="docutils literal notranslate"><span class="pre">ENOENT</span></code> if the object does not exist</p>
<p>For a redirect manifest, copies data to head.</p>
<p>TODO: Promote on a redirect object needs to clear the redirect.</p>
<p>For a chunked manifest, reads all MISSING extents into the base pool,
subsequent reads and writes will be served from the base pool.</p>
<p>Implementation Note: For a chunked manifest, calls <code class="docutils literal notranslate"><span class="pre">start_copy</span></code> on itself.  The
resulting <code class="docutils literal notranslate"><span class="pre">copy_get</span></code> operation will issue reads which will then be redirected by
the normal manifest read machinery.</p>
<p>Does not set the <code class="docutils literal notranslate"><span class="pre">user_modify</span></code> flag.</p>
<p>Future work will involve adding support for specifying a <code class="docutils literal notranslate"><span class="pre">clone_id</span></code>.</p>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">unset-manifest</span></code></p>
<p>Unset the manifest info in the object that has manifest.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">void</span> <span class="n">unset_manifest</span><span class="p">();</span>

<span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">base_pool</span> <span class="n">unset</span><span class="o">-</span><span class="n">manifest</span> <span class="o">&lt;</span><span class="n">obj</span><span class="o">-</span><span class="n">name</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>Clears manifest chunks or redirect.  Lazily releases references, may
leak.</p>
<p><code class="docutils literal notranslate"><span class="pre">do_osd_ops</span></code> seems not to include it in the <code class="docutils literal notranslate"><span class="pre">user_modify=false</span></code> <code class="docutils literal notranslate"><span class="pre">ignorelist</span></code>,
and so will trigger a snapshot.  Note, this will be true even for a
redirect though <code class="docutils literal notranslate"><span class="pre">SET_REDIRECT</span></code> does not flip <code class="docutils literal notranslate"><span class="pre">user_modify</span></code>.  This should
be fixed -- <code class="docutils literal notranslate"><span class="pre">unset-manifest</span></code> should not be a <code class="docutils literal notranslate"><span class="pre">user_modify</span></code>.</p>
</li>
<li><p><code class="docutils literal notranslate"><span class="pre">tier-flush</span></code></p>
<p>Flush the object which has chunks to the chunk pool.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">void</span> <span class="n">tier_flush</span><span class="p">();</span>

<span class="n">rados</span> <span class="o">-</span><span class="n">p</span> <span class="n">base_pool</span> <span class="n">tier</span><span class="o">-</span><span class="n">flush</span> <span class="o">&lt;</span><span class="n">obj</span><span class="o">-</span><span class="n">name</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>Included in the <code class="docutils literal notranslate"><span class="pre">user_modify=false</span></code> <code class="docutils literal notranslate"><span class="pre">ignorelist</span></code>, does not trigger a clone.</p>
<p>Does not evict the extents.</p>
</li>
</ul>
</section>
<section id="ceph-dedup-tool">
<h2>ceph-dedup-tool<a class="headerlink" href="#ceph-dedup-tool" title="Permalink to this heading"></a></h2>
<p><code class="docutils literal notranslate"><span class="pre">ceph-dedup-tool</span></code> has two features: finding an optimal chunk offset for dedup chunking
and fixing the reference count (see <code class="docutils literal notranslate"><span class="pre">./refcount.rst</span></code>).</p>
<ul>
<li><p>Find an optimal chunk offset</p>
<ol class="loweralpha simple">
<li><p>Fixed chunk</p></li>
</ol>
<p>To find out a fixed chunk length, you need to run the following command many
times while changing the <code class="docutils literal notranslate"><span class="pre">chunk_size</span></code>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool --op estimate --pool $POOL --chunk-size chunk_size
  --chunk-algorithm fixed --fingerprint-algorithm sha1|sha256|sha512
</pre></div>
</div>
<ol class="loweralpha simple" start="2">
<li><p>Rabin chunk(Rabin-Karp algorithm)</p></li>
</ol>
<p>Rabin-Karp is a string-searching algorithm based
on a rolling hash. But a rolling hash is not enough to do deduplication because
we don’t know the chunk boundary. So, we need content-based slicing using
a rolling hash for content-defined chunking.
The current implementation uses the simplest approach: look for chunk boundaries
by inspecting the rolling hash for pattern (like the
lower N bits are all zeroes).</p>
<p>Users who want to use deduplication need to find an ideal chunk offset.
To find out ideal chunk offset, users should discover
the optimal configuration for their data workload via <code class="docutils literal notranslate"><span class="pre">ceph-dedup-tool</span></code>.
This information will then be used for object chunking through
the <code class="docutils literal notranslate"><span class="pre">set-chunk</span></code> API.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool --op estimate --pool $POOL --min-chunk min_size
  --chunk-algorithm rabin --fingerprint-algorithm rabin
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">ceph-dedup-tool</span></code> has many options to utilize <code class="docutils literal notranslate"><span class="pre">rabin</span> <span class="pre">chunk</span></code>.
These are options for <code class="docutils literal notranslate"><span class="pre">rabin</span> <span class="pre">chunk</span></code>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>           --mod-prime &lt;uint64_t&gt;
           --rabin-prime &lt;uint64_t&gt;
           --pow &lt;uint64_t&gt;
           --chunk-mask-bit &lt;uint32_t&gt;
           --window-size &lt;uint32_t&gt;
           --min-chunk &lt;uint32_t&gt;
           --max-chunk &lt;uint64_t&gt;

Users need to refer following equation to use above options for ``rabin chunk``. ::

           rabin_hash =
             (rabin_hash * rabin_prime + new_byte - old_byte * pow) % (mod_prime)
</pre></div>
</div>
<ol class="loweralpha simple" start="3">
<li><p>Fixed chunk vs content-defined chunk</p></li>
</ol>
<p>Content-defined chunking may or not be optimal solution.
For example,</p>
<p>Data chunk <code class="docutils literal notranslate"><span class="pre">A</span></code> : <code class="docutils literal notranslate"><span class="pre">abcdefgabcdefgabcdefg</span></code></p>
<p>Let’s think about Data chunk <code class="docutils literal notranslate"><span class="pre">A</span></code>’s deduplication. The ideal chunk offset is
from <code class="docutils literal notranslate"><span class="pre">1</span></code> to <code class="docutils literal notranslate"><span class="pre">7</span></code> (<code class="docutils literal notranslate"><span class="pre">abcdefg</span></code>). So, if we use fixed chunk, <code class="docutils literal notranslate"><span class="pre">7</span></code> is optimal chunk length.
But, in the case of content-based slicing, the optimal chunk length
could not be found (dedup ratio will not be 100%).
Because we need to find optimal parameter such
as boundary bit, window size and prime value. This is as easy as fixed chunk.
But, content defined chunking is very effective in the following case.</p>
<blockquote>
<div><p>Data chunk <code class="docutils literal notranslate"><span class="pre">B</span></code> : <code class="docutils literal notranslate"><span class="pre">abcdefgabcdefgabcdefg</span></code></p>
<p>Data chunk <code class="docutils literal notranslate"><span class="pre">C</span></code> : <code class="docutils literal notranslate"><span class="pre">Tabcdefgabcdefgabcdefg</span></code></p>
</div></blockquote>
</li>
<li><p>Fix reference count</p>
<p>The key idea behind of reference counting for dedup is false-positive, which means
<code class="docutils literal notranslate"><span class="pre">(manifest</span> <span class="pre">object</span> <span class="pre">(no</span> <span class="pre">ref),,</span> <span class="pre">chunk</span> <span class="pre">object(has</span> <span class="pre">ref))</span></code> happen instead of
<code class="docutils literal notranslate"><span class="pre">(manifest</span> <span class="pre">object</span> <span class="pre">(has</span> <span class="pre">ref),</span> <span class="pre">chunk</span> <span class="pre">1(no</span> <span class="pre">ref))</span></code>.
To fix such inconsistencies, <code class="docutils literal notranslate"><span class="pre">ceph-dedup-tool</span></code> supports <code class="docutils literal notranslate"><span class="pre">chunk_scrub</span></code>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>ceph-dedup-tool --op chunk_scrub --chunk_pool $CHUNK_POOL
</pre></div>
</div>
</li>
</ul>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../log_based_pg/" class="btn btn-neutral float-left" title="Log Based PG" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../map_message_handling/" class="btn btn-neutral float-right" title="Map and PG Message handling" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>